.ipynb .pdf

repository open issue suggest edit

On this page

1. Trajectory Divergence

from IPython.display import display
from math import cos, sin, pi, sqrt
import random
import pickle as pkl
import numpy as np
import pandas as pd

import plotly.graph_objects as go
import plotly.express as px
import plotly.offline as pyo
pyo.init_notebook_mode()
from ipythonblocks import BlockGrid
from webcolors import name_to_rgb

import warnings
warnings.filterwarnings('ignore')

Trajectory data is saved as a pandas dataframe in trajectories.pkl. Let’s import it and modify it for our needs.

We first need to calculate the mean trajectories for each clip, averaged across all participants. This will be stored in a new dataframe for just mean trajectories.

We also want to define where the mean trajectory ends, but just choosing one point would be too noisy. Trajectories eventually stagnate around a certain space for multiple time steps, so we can define this space as a sphere. We’ll define the sphere’s center point as the mean of a group of points near the end of the trajectory, where “end” is based on time. The radius will then be set as the farthest distance from the mean point to any individual point. We can then calculate the density of trajectory points inside the sphere. The goal is to find a sphere with the highest density. To accomplish this, the points included in the group is tested between the last 10 points to all points in the trajectory. The mean point and radius are then stored in the mean dataframe.

We can also store some information about each clip into a new dataframe. This info includes clip ID, clip name, and clip length.

The three dataframes can be stored in a .pkl file for easy importing without having to redo calculations. Let’s display the three dataframes we’ve created.

# import data
with open('trajectories.pkl', 'rb') as f:
    data = pkl.load(f)
traj_df = data['traj_df'] # pandas dataframe with 3D coordinates, time, particpant id (pid), clip id (clip) and  clip name as columns
clip_len = data['clip_len'] # array consisting number of time points indexed by clip_id

# add clip length to traj_df, then reorder traj_df
traj_df['clip_len'] = traj_df['clip'].transform(lambda x: clip_len[x])
traj_df = traj_df[['clip','clip_name','clip_len','pid','time','x','y','z']]

# create new df for each clip's mean trajectories
mean_df = traj_df.drop(columns=['pid']).groupby(['clip','clip_name','clip_len','time']).agg(np.mean).reset_index()

# calculate mean trajectory ends
end_x = np.array([])
end_y = np.array([])
end_z = np.array([])
end_r = np.array([])
for clip, group in mean_df.groupby('clip'):

    max_density = 0
    r = 0
    curr_clip_len = group['clip_len'].iloc[0]
    
    # iterate through number of points to include in end
    for num_points in range(10,curr_clip_len+1):
        
        # calculate mean of points
        temp_df = group.drop(columns=['clip','clip_name','clip_len','time']).iloc[curr_clip_len-num_points : curr_clip_len-1]
        mean = temp_df.agg(np.mean)
        
        # calculate min radius that includes all points
        max_r = -1
        for i,point in temp_df.iterrows():
            curr_r = sqrt((mean['x']-point['x'])**2 + (mean['y']-point['y'])**2 + (mean['z']-point['z'])**2)
            if (curr_r > max_r):
                max_r = curr_r
        
        # calculate greatest density of points in sphere
        num_points_in = 0
        for i,point in group.drop(columns=['clip','clip_name','clip_len','time']).iterrows():
            if (sqrt((mean['x']-point['x'])**2 + (mean['y']-point['y'])**2 + (mean['z']-point['z'])**2) <= max_r):
                num_points_in += 1

        curr_density = num_points_in**3 / (4/3*pi*max_r**3)
        if (curr_density > max_density):
            max_density = curr_density
            r = max_r
            best_end = mean
            
    # add end data to arrays
    end_x = np.concatenate((end_x, np.ones(curr_clip_len)*best_end['x']))
    end_y = np.concatenate((end_y, np.ones(curr_clip_len)*best_end['y']))
    end_z = np.concatenate((end_z, np.ones(curr_clip_len)*best_end['z']))
    end_r = np.concatenate((end_r, np.ones(curr_clip_len)*r))

mean_df['end_x'] = end_x
mean_df['end_y'] = end_y
mean_df['end_z'] = end_z
mean_df['end_r'] = end_r

# create new df for unique clip ids, names, and lengths
clipdata_df = pd.DataFrame({'clip':np.arange(0,len(clip_len)),
                            'clip_name':traj_df.clip_name.unique(),
                            'clip_len':clip_len},
                           columns=['clip','clip_name','clip_len'])

# save dataframes
with open("trajectories_updated.pkl", "wb") as f:
    pkl.dump({'traj_df':traj_df, 'mean_df':mean_df, 'clipdata_df':clipdata_df}, f)

#load
with open('trajectories_updated.pkl', 'rb') as f:
    data = pkl.load(f)
traj_df = data['traj_df']
mean_df = data['mean_df']
clipdata_df = data['clipdata_df']

# display dataframes
print('traj_df:',end='')
display(traj_df)
print('\nmean_df:',end='')
display(mean_df)
print('\nclipdata_df:',end='')
display(clipdata_df)

traj_df:

	clip	clip_name	clip_len	pid	time	x	y	z
0	0	testretest	84	1	0	-0.068375	0.292656	0.076036
1	0	testretest	84	1	1	-0.560828	0.290854	0.095379
2	0	testretest	84	1	2	0.248541	-0.024260	-0.019393
3	0	testretest	84	1	3	-0.021169	0.253559	1.618106
4	0	testretest	84	1	4	-0.218407	0.420255	2.193483
...	...	...	...	...	...	...	...	...
245399	14	starwars	256	76	251	-6.074567	-14.429848	13.255661
245400	14	starwars	256	76	252	-5.333982	-15.487228	15.741982
245401	14	starwars	256	76	253	-5.229411	-14.917367	16.472929
245402	14	starwars	256	76	254	-4.298551	-12.905822	15.564515
245403	14	starwars	256	76	255	-3.862932	-14.278425	16.305193

245404 rows × 8 columns

mean_df:

	clip	clip_name	clip_len	time	x	y	z	end_x	end_y	end_z	end_r
0	0	testretest	84	0	-0.535419	-0.584089	0.527666	-2.512531	-16.690886	14.976014	0.625368
1	0	testretest	84	1	-0.828858	-1.073595	0.652328	-2.512531	-16.690886	14.976014	0.625368
2	0	testretest	84	2	-0.948996	-1.365094	0.686838	-2.512531	-16.690886	14.976014	0.625368
3	0	testretest	84	3	-1.016553	-1.504698	0.718371	-2.512531	-16.690886	14.976014	0.625368
4	0	testretest	84	4	-1.068053	-1.757247	0.923804	-2.512531	-16.690886	14.976014	0.625368
...	...	...	...	...	...	...	...	...	...	...	...
2972	14	starwars	256	251	7.020762	-20.638204	2.571888	7.565658	-20.862614	2.771892	1.202724
2973	14	starwars	256	252	6.981013	-20.744694	2.663826	7.565658	-20.862614	2.771892	1.202724
2974	14	starwars	256	253	7.065092	-20.584518	2.731824	7.565658	-20.862614	2.771892	1.202724
2975	14	starwars	256	254	7.200529	-20.280288	2.620943	7.565658	-20.862614	2.771892	1.202724
2976	14	starwars	256	255	7.261687	-20.207359	2.578671	7.565658	-20.862614	2.771892	1.202724

2977 rows × 11 columns

clipdata_df:

	clip	clip_name	clip_len
0	0	testretest	84
1	1	twomen	245
2	2	bridgeville	222
3	3	pockets	189
4	4	overcome	65
5	5	inception	227
6	6	socialnet	260
7	7	oceans	250
8	8	flower	181
9	9	hotel	186
10	10	garden	205
11	11	dreary	143
12	12	homealone	233
13	13	brokovich	231
14	14	starwars	256

Let’s create a colorscale so we can easily visualize the 15 clips on one graph.

grid = BlockGrid(15,1,fill=(0,0,0))
grid.block_size = 50
grid.lines_on = False

colors = ['slategray','sienna','darkred','crimson','darkorange','darkgoldenrod','darkkhaki','mediumseagreen','darkgreen','darkcyan','cornflowerblue','mediumblue','blueviolet','purple','hotpink']
i = 0
for block in grid:
    color = name_to_rgb(colors[i])
    block.set_colors(color[0],color[1],color[2])
    i+=1
    
grid.show()

Let’s plot the mean trajectories and their ends.

plotly_data = []

for clip, clip_name in enumerate(clipdata_df['clip_name']):

    # mean trajectories
    temp_df = mean_df[(mean_df.clip_name==clip_name)]
    custom_df = temp_df[['time','clip_len']]
    custom_df['clip_len'] = custom_df['clip_len']-1
    mean_traj = go.Scatter3d(
        x=temp_df['x'],
        y=temp_df['y'],
        z=temp_df['z'],
        customdata=custom_df,
        mode='markers+lines',
        marker={'size':2, 'color': colors[clip]},
        line={'width':4, 'color': colors[clip]},
        name=clip_name,
        hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}/%{customdata[1]}')
    plotly_data.append(mean_traj)
    
    # end area
    theta = np.linspace(0,2*pi,50)
    phi = np.linspace(0,pi,50)
    r = temp_df['end_r'].iloc[0]
    x = r*np.outer(np.cos(theta),np.sin(phi)) + temp_df['end_x'].iloc[0]
    y = r*np.outer(np.sin(theta),np.sin(phi)) + temp_df['end_y'].iloc[0]
    z = r*np.outer(np.ones(50),np.cos(phi)) + temp_df['end_z'].iloc[0]
    sphere = go.Surface(
        x=x,
        y=y,
        z=z,
        opacity=0.3,
        hoverinfo='skip',
        showscale=False,
        colorscale=[colors[clip],colors[clip]])
    plotly_data.append(sphere)

# formatting
plotly_layout = go.Layout(showlegend=True, 
                          autosize=False,
                          width=800, 
                          height=800,
                          margin={'l':0, 'r':0, 't':40, 'b':0},
                          legend={'orientation':'h',
                                  'itemsizing':'constant',
                                  'xanchor':'center',
                                  'yanchor':'bottom',
                                  'y':-0.1,
                                  'x':0.5},
                          title={'text':'Mean Trajectory Per Clip',
                                 'xanchor':'center',
                                 'yanchor':'top',
                                 'x':0.5,
                                 'y':0.98})

plotly_config = {'displaylogo':False,
                 'modeBarButtonsToRemove': ['resetCameraLastSave3d','orbitRotation']}

fig_traj = go.Figure(data=plotly_data, layout=plotly_layout)
fig_traj.show(config=plotly_config)

Now let’s look at two clips that have similar mean trajectories, “ocean” and “brokovich” in this case. We’ll plot their mean trajectories (represented as points connected by lines) as well as all of their individual trajectories (represented as individual unconnected points).

plotly_data = []



# OCEANS

## mean trajectory 
temp_df = mean_df[(mean_df.clip_name=='oceans')]
temp_df['clip_len'] = temp_df['clip'].transform(lambda x: clip_len[x]-1)
mean_traj = go.Scatter3d(
    x=temp_df['x'],
    y=temp_df['y'],
    z=temp_df['z'],
    customdata=temp_df[['time','clip_len']],
    mode='markers+lines',
    marker={'size':2, 'color':'blue'},
    line={'width':4, 'color':'blue'},
    name='oceans',
    legendgroup='oceans',
    hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}/%{customdata[1]}')
plotly_data.append(mean_traj)

theta = np.linspace(0,2*pi,50)
phi = np.linspace(0,pi,50)
r = temp_df['end_r'].iloc[0]
x = r*np.outer(np.cos(theta),np.sin(phi)) + temp_df['end_x'].iloc[0]
y = r*np.outer(np.sin(theta),np.sin(phi)) + temp_df['end_y'].iloc[0]
z = r*np.outer(np.ones(50),np.cos(phi)) + temp_df['end_z'].iloc[0]
sphere = go.Surface(
    x=x,
    y=y,
    z=z,
    opacity=0.3,
    hoverinfo='skip',
    showscale=False,
    colorscale=['blue','blue'])
plotly_data.append(sphere)

## individual trajectories
temp_df = traj_df[(traj_df.clip_name=='oceans')]
pid_traj = go.Scatter3d(
    x=temp_df['x'],
    y=temp_df['y'],
    z=temp_df['z'],
    customdata=temp_df['time'],
    mode='markers',
    marker={'size':0.5, 'color':'blue'},
    opacity=0.5,
    name='oceans',
    legendgroup='oceans',
    showlegend=False,
    hoverinfo='skip')
plotly_data.append(pid_traj)



# BROKOVICH

## mean trajectory 
temp_df = mean_df[(mean_df.clip_name=='brokovich')]
temp_df['clip_len'] = temp_df['clip'].transform(lambda x: clip_len[x]-1)
mean_traj = go.Scatter3d(
    x=temp_df['x'],
    y=temp_df['y'],
    z=temp_df['z'],
    customdata=temp_df[['time','clip_len']],
    mode='markers+lines',
    marker={'size':2, 'color':'red'},
    line={'width':4, 'color':'red'},
    name='brokovich',
    legendgroup='brokovich',
    hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}/%{customdata[1]}')
plotly_data.append(mean_traj)

theta = np.linspace(0,2*pi,50)
phi = np.linspace(0,pi,50)
r = temp_df['end_r'].iloc[0]
x = r*np.outer(np.cos(theta),np.sin(phi)) + temp_df['end_x'].iloc[0]
y = r*np.outer(np.sin(theta),np.sin(phi)) + temp_df['end_y'].iloc[0]
z = r*np.outer(np.ones(50),np.cos(phi)) + temp_df['end_z'].iloc[0]
sphere = go.Surface(
    x=x,
    y=y,
    z=z,
    opacity=0.3,
    hoverinfo='skip',
    showscale=False,
    colorscale=['red','red'])
plotly_data.append(sphere)

## individual trajectories
temp_df = traj_df[(traj_df.clip_name=='brokovich')]
pid_traj = go.Scatter3d(
    x=temp_df['x'],
    y=temp_df['y'],
    z=temp_df['z'],
    customdata=temp_df['time'],
    mode='markers',
    marker={'size':0.5, 'color':'red'},
    opacity=0.5,
    name='brokovich',
    legendgroup='brokovich',
    showlegend=False,
    hoverinfo='skip')
plotly_data.append(pid_traj)



# formatting
plotly_layout = go.Layout(showlegend=True, 
                          autosize=False,
                          width=800, 
                          height=600,
                          margin={'l':0, 'r':0, 't':35, 'b':0},
                          legend={'orientation':'h',
                                  'itemsizing':'constant',
                                  'xanchor':'center',
                                  'yanchor':'bottom',
                                  'y':-0.055,
                                  'x':0.5},
                          title={'text':'Mean and Individual Trajectories for \"oceans\" and \"brokovich\"',
                                 'xanchor':'center',
                                 'yanchor':'top',
                                 'x':0.5,
                                 'y':0.98})

plotly_config = {'displaylogo':False,
                 'modeBarButtonsToRemove': ['resetCameraLastSave3d','orbitRotation']}

fig_traj = go.Figure(data=plotly_data, layout=plotly_layout)
fig_traj.show(config=plotly_config)

We see that the majority of individual points lie in clouds near the end of their corresponding mean trajectories, with many other points surrounding the mean trajectories. This behavior indicates that most individual trajectories follow a similar path and end in a similar location, especially since the mean trajectories themselves stay in a relatively close clump for the latter half of the clips.

However, there are a fair amount of points that don’t follow this trend. Let’s plot the endpoints of all individual trajectories to see where they finish at the end of the clip.

plotly_data = []



# OCEANS

## mean trajectory 
temp_df = mean_df[(mean_df.clip_name=='oceans')]
temp_df['clip_len'] = temp_df['clip'].transform(lambda x: clip_len[x]-1)
mean_traj = go.Scatter3d(
    x=temp_df['x'],
    y=temp_df['y'],
    z=temp_df['z'],
    customdata=temp_df[['time','clip_len']],
    mode='markers+lines',
    marker={'size':2, 'color':'blue'},
    line={'width':4, 'color':'blue'},
    name='oceans',
    legendgroup='oceans',
    hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}/%{customdata[1]}<br>mean')
plotly_data.append(mean_traj)

theta = np.linspace(0,2*pi,50)
phi = np.linspace(0,pi,50)
r = temp_df['end_r'].iloc[0]
x = r*np.outer(np.cos(theta),np.sin(phi)) + temp_df['end_x'].iloc[0]
y = r*np.outer(np.sin(theta),np.sin(phi)) + temp_df['end_y'].iloc[0]
z = r*np.outer(np.ones(50),np.cos(phi)) + temp_df['end_z'].iloc[0]
sphere = go.Surface(
    x=x,
    y=y,
    z=z,
    opacity=0.3,
    hoverinfo='skip',
    showscale=False,
    colorscale=['blue','blue'])
plotly_data.append(sphere)



## individual trajectories
temp_df = traj_df[(traj_df.clip_name=='oceans') & (traj_df.time==clipdata_df[clipdata_df.clip_name=='oceans']['clip_len'].iloc[0]-1)]
pid_traj = go.Scatter3d(
    x=temp_df['x'],
    y=temp_df['y'],
    z=temp_df['z'],
    customdata=temp_df[['time','pid']],
    mode='markers',
    marker={'size':4, 'color':'blue'},
    opacity=0.5,
    name='oceans',
    legendgroup='oceans',
    showlegend=False,
    hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}<br>pid: %{customdata[1]}')
plotly_data.append(pid_traj)



# # BROKOVICH

## mean trajectory 
temp_df = mean_df[(mean_df.clip_name=='brokovich')]
temp_df['clip_len'] = temp_df['clip'].transform(lambda x: clip_len[x]-1)
mean_traj = go.Scatter3d(
    x=temp_df['x'],
    y=temp_df['y'],
    z=temp_df['z'],
    customdata=temp_df[['time','clip_len']],
    mode='markers+lines',
    marker={'size':2, 'color':'red'},
    line={'width':4, 'color':'red'},
    name='brokovich',
    legendgroup='brokovich',
    hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}/%{customdata[1]}<br>mean')
plotly_data.append(mean_traj)

theta = np.linspace(0,2*pi,50)
phi = np.linspace(0,pi,50)
r = temp_df['end_r'].iloc[0]
x = r*np.outer(np.cos(theta),np.sin(phi)) + temp_df['end_x'].iloc[0]
y = r*np.outer(np.sin(theta),np.sin(phi)) + temp_df['end_y'].iloc[0]
z = r*np.outer(np.ones(50),np.cos(phi)) + temp_df['end_z'].iloc[0]
sphere = go.Surface(
    x=x,
    y=y,
    z=z,
    opacity=0.3,
    hoverinfo='skip',
    showscale=False,
    colorscale=['red','red'])
plotly_data.append(sphere)

## individual trajectories
temp_df = traj_df[(traj_df.clip_name=='brokovich') & (traj_df.time==clipdata_df[clipdata_df.clip_name=='brokovich']['clip_len'].iloc[0]-1)]
pid_traj = go.Scatter3d(
    x=temp_df['x'],
    y=temp_df['y'],
    z=temp_df['z'],
    customdata=temp_df[['time','pid']],
    mode='markers',
    marker={'size':4, 'color':'red'},
    opacity=0.5,
    name='brokovich',
    legendgroup='brokovich',
    showlegend=False,
    hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}<br>pid: %{customdata[1]}')
plotly_data.append(pid_traj)



# formatting
plotly_layout = go.Layout(showlegend=True, 
                          autosize=False,
                          width=800, 
                          height=600,
                          margin={'l':0, 'r':0, 't':35, 'b':0},
                          legend={'orientation':'h',
                                  'itemsizing':'constant',
                                  'xanchor':'center',
                                  'yanchor':'bottom',
                                  'y':-0.055,
                                  'x':0.5},
                          title={'text':'Individual Trajectory Endpoints for \"oceans\" and \"brokovich\"',
                                 'xanchor':'center',
                                 'yanchor':'top',
                                 'x':0.5,
                                 'y':0.98})

plotly_config = {'displaylogo':False,
                 'modeBarButtonsToRemove': ['resetCameraLastSave3d','orbitRotation']}

fig_traj = go.Figure(data=plotly_data, layout=plotly_layout)
fig_traj.show(config=plotly_config)

It appears that most trajectories end at a similar location as the mean trajectories. There aren’t any other noticeable groups, so let’s take a look at some of the trajectories that fall into the cloud around the mean trajectories.

plotly_data = []



# OCEANS

## mean trajectory 
temp_df = mean_df[(mean_df.clip_name=='oceans')]
temp_df['clip_len'] = temp_df['clip'].transform(lambda x: clip_len[x]-1)
mean_traj = go.Scatter3d(
    x=temp_df['x'],
    y=temp_df['y'],
    z=temp_df['z'],
    customdata=temp_df[['time','clip_len']],
    mode='markers+lines',
    marker={'size':2, 'color':'blue'},
    line={'width':4, 'color':'blue'},
    name='oceans',
    legendgroup='oceans',
    hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}/%{customdata[1]}<br>mean')
plotly_data.append(mean_traj)

theta = np.linspace(0,2*pi,50)
phi = np.linspace(0,pi,50)
r = temp_df['end_r'].iloc[0]
x = r*np.outer(np.cos(theta),np.sin(phi)) + temp_df['end_x'].iloc[0]
y = r*np.outer(np.sin(theta),np.sin(phi)) + temp_df['end_y'].iloc[0]
z = r*np.outer(np.ones(50),np.cos(phi)) + temp_df['end_z'].iloc[0]
sphere = go.Surface(
    x=x,
    y=y,
    z=z,
    opacity=0.3,
    hoverinfo='skip',
    showscale=False,
    colorscale=['blue','blue'])
plotly_data.append(sphere)

## individual trajectories
for pid in ([32,57,73]):
    temp_df = traj_df[(traj_df.clip_name=='oceans') & (traj_df.pid==pid)]
    temp_df['clip_len'] = temp_df['clip'].transform(lambda x: clip_len[x]-1)
    pid_traj = go.Scatter3d(
        x=temp_df['x'],
        y=temp_df['y'],
        z=temp_df['z'],
        customdata=temp_df[['time','clip_len','pid']],
        mode='markers+lines',
        marker={'size':1, 'color':'blue'},
        opacity=0.5,
        name='oceans',
        legendgroup='oceans',
        showlegend=False,
        hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}/%{customdata[1]}<br>pid: %{customdata[2]}')
    plotly_data.append(pid_traj)
    
    temp_df = traj_df[(traj_df.clip_name=='oceans') & (traj_df.pid==pid) & (traj_df.time==clipdata_df[clipdata_df.clip_name=='oceans']['clip_len'].iloc[0]-1)]
    pid_traj = go.Scatter3d(
        x=temp_df['x'],
        y=temp_df['y'],
        z=temp_df['z'],
        customdata=temp_df[['time','pid']],
        mode='markers',
        marker={'size':8, 'color':'blue'},
        opacity=0.7,
        name='oceans',
        legendgroup='oceans',
        showlegend=False,
        hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}<br>pid: %{customdata[1]}')
    plotly_data.append(pid_traj)



# BROKOVICH

## mean trajectory 
temp_df = mean_df[(mean_df.clip_name=='brokovich')]
temp_df['clip_len'] = temp_df['clip'].transform(lambda x: clip_len[x]-1)
mean_traj = go.Scatter3d(
    x=temp_df['x'],
    y=temp_df['y'],
    z=temp_df['z'],
    customdata=temp_df[['time','clip_len']],
    mode='markers+lines',
    marker={'size':2, 'color':'red'},
    line={'width':4, 'color':'red'},
    name='brokovich',
    legendgroup='brokovich',
    hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}/%{customdata[1]}<br>mean')
plotly_data.append(mean_traj)

theta = np.linspace(0,2*pi,50)
phi = np.linspace(0,pi,50)
r = temp_df['end_r'].iloc[0]
x = r*np.outer(np.cos(theta),np.sin(phi)) + temp_df['end_x'].iloc[0]
y = r*np.outer(np.sin(theta),np.sin(phi)) + temp_df['end_y'].iloc[0]
z = r*np.outer(np.ones(50),np.cos(phi)) + temp_df['end_z'].iloc[0]
sphere = go.Surface(
    x=x,
    y=y,
    z=z,
    opacity=0.3,
    hoverinfo='skip',
    showscale=False,
    colorscale=['red','red'])
plotly_data.append(sphere)

## individual trajectories
for pid in ([2,20,72]):
    temp_df = traj_df[(traj_df.clip_name=='brokovich') & (traj_df.pid==pid)]
    temp_df['clip_len'] = temp_df['clip'].transform(lambda x: clip_len[x]-1)
    pid_traj = go.Scatter3d(
        x=temp_df['x'],
        y=temp_df['y'],
        z=temp_df['z'],
        customdata=temp_df[['time','clip_len','pid']],
        mode='markers+lines',
        marker={'size':1, 'color':'red'},
        opacity=0.5,
        name='brokovich',
        legendgroup='brokovich',
        showlegend=False,
        hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}/%{customdata[1]}<br>pid: %{customdata[2]}')
    plotly_data.append(pid_traj)
    
    temp_df = traj_df[(traj_df.clip_name=='brokovich') & (traj_df.pid==pid) & (traj_df.time==clipdata_df[clipdata_df.clip_name=='brokovich']['clip_len'].iloc[0]-1)]
    pid_traj = go.Scatter3d(
        x=temp_df['x'],
        y=temp_df['y'],
        z=temp_df['z'],
        customdata=temp_df[['time','pid']],
        mode='markers',
        marker={'size':8, 'color':'red'},
        opacity=0.7,
        name='brokovich',
        legendgroup='brokovich',
        showlegend=False,
        hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}<br>pid: %{customdata[1]}')
    plotly_data.append(pid_traj)
    



# formatting
plotly_layout = go.Layout(showlegend=True, 
                          autosize=False,
                          width=800, 
                          height=600,
                          margin={'l':0, 'r':0, 't':35, 'b':0},
                          legend={'orientation':'h',
                                  'itemsizing':'constant',
                                  'xanchor':'center',
                                  'yanchor':'bottom',
                                  'y':-0.055,
                                  'x':0.5},
                          title={'text':'Mean-Like Trajectories for \"oceans\" and \"brokovich\"',
                                 'xanchor':'center',
                                 'yanchor':'top',
                                 'x':0.5,
                                 'y':0.98})

plotly_config = {'displaylogo':False,
                 'modeBarButtonsToRemove': ['resetCameraLastSave3d','orbitRotation']}

fig_traj = go.Figure(data=plotly_data, layout=plotly_layout)
fig_traj.show(config=plotly_config)

Just from looking at a few individual trajectories, we can notice that some trajectories do meander a fair amount before ending at the mean trajectory cloud. Notable examples include oceans 32, oceans 57, brokovich 20, and brokovich 72. Some other trajectories such as oceans 73 and brokovich 2 do follow a more similar path to the mean trajectory, but they still exhibit slight meandering.

We should also take a look at some trajectories that ended at a completely different point from the mean trajectories.

plotly_data = []



# OCEANS

## mean trajectory 
temp_df = mean_df[(mean_df.clip_name=='oceans')]
temp_df['clip_len'] = temp_df['clip'].transform(lambda x: clip_len[x]-1)
mean_traj = go.Scatter3d(
    x=temp_df['x'],
    y=temp_df['y'],
    z=temp_df['z'],
    customdata=temp_df[['time','clip_len']],
    mode='markers+lines',
    marker={'size':2, 'color':'blue'},
    line={'width':4, 'color':'blue'},
    name='oceans',
    legendgroup='oceans',
    hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}/%{customdata[1]}<br>mean')
plotly_data.append(mean_traj)

theta = np.linspace(0,2*pi,50)
phi = np.linspace(0,pi,50)
r = temp_df['end_r'].iloc[0]
x = r*np.outer(np.cos(theta),np.sin(phi)) + temp_df['end_x'].iloc[0]
y = r*np.outer(np.sin(theta),np.sin(phi)) + temp_df['end_y'].iloc[0]
z = r*np.outer(np.ones(50),np.cos(phi)) + temp_df['end_z'].iloc[0]
sphere = go.Surface(
    x=x,
    y=y,
    z=z,
    opacity=0.3,
    hoverinfo='skip',
    showscale=False,
    colorscale=['blue','blue'])
plotly_data.append(sphere)

## individual trajectories
for pid in ([17,18]):
    temp_df = traj_df[(traj_df.clip_name=='oceans') & (traj_df.pid==pid)]
    temp_df['clip_len'] = temp_df['clip'].transform(lambda x: clip_len[x]-1)
    pid_traj = go.Scatter3d(
        x=temp_df['x'],
        y=temp_df['y'],
        z=temp_df['z'],
        customdata=temp_df[['time','clip_len','pid']],
        mode='markers+lines',
        marker={'size':1, 'color':'blue'},
        opacity=0.5,
        name='oceans',
        legendgroup='oceans',
        showlegend=False,
        hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}/%{customdata[1]}<br>pid: %{customdata[2]}')
    plotly_data.append(pid_traj)
    
    temp_df = traj_df[(traj_df.clip_name=='oceans') & (traj_df.pid==pid) & (traj_df.time==clipdata_df[clipdata_df.clip_name=='oceans']['clip_len'].iloc[0]-1)]
    pid_traj = go.Scatter3d(
        x=temp_df['x'],
        y=temp_df['y'],
        z=temp_df['z'],
        customdata=temp_df[['time','pid']],
        mode='markers',
        marker={'size':8, 'color':'blue'},
        opacity=0.5,
        name='oceans',
        legendgroup='oceans',
        showlegend=False,
        hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}<br>pid: %{customdata[1]}')
    plotly_data.append(pid_traj)



# BROKOVICH

## mean trajectory 
temp_df = mean_df[(mean_df.clip_name=='brokovich')]
temp_df['clip_len'] = temp_df['clip'].transform(lambda x: clip_len[x]-1)
mean_traj = go.Scatter3d(
    x=temp_df['x'],
    y=temp_df['y'],
    z=temp_df['z'],
    customdata=temp_df[['time','clip_len']],
    mode='markers+lines',
    marker={'size':2, 'color':'red'},
    line={'width':4, 'color':'red'},
    name='brokovich',
    legendgroup='brokovich',
    hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}/%{customdata[1]}<br>mean')
plotly_data.append(mean_traj)

theta = np.linspace(0,2*pi,50)
phi = np.linspace(0,pi,50)
r = temp_df['end_r'].iloc[0]
x = r*np.outer(np.cos(theta),np.sin(phi)) + temp_df['end_x'].iloc[0]
y = r*np.outer(np.sin(theta),np.sin(phi)) + temp_df['end_y'].iloc[0]
z = r*np.outer(np.ones(50),np.cos(phi)) + temp_df['end_z'].iloc[0]
sphere = go.Surface(
    x=x,
    y=y,
    z=z,
    opacity=0.3,
    hoverinfo='skip',
    showscale=False,
    colorscale=['red','red'])
plotly_data.append(sphere)

## individual trajectories
for pid in ([49,56]):
    temp_df = traj_df[(traj_df.clip_name=='brokovich') & (traj_df.pid==pid)]
    temp_df['clip_len'] = temp_df['clip'].transform(lambda x: clip_len[x]-1)
    pid_traj = go.Scatter3d(
        x=temp_df['x'],
        y=temp_df['y'],
        z=temp_df['z'],
        customdata=temp_df[['time','clip_len','pid']],
        mode='markers+lines',
        marker={'size':1, 'color':'red'},
        opacity=0.5,
        name='brokovich',
        legendgroup='brokovich',
        showlegend=False,
        hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}/%{customdata[1]}<br>pid: %{customdata[2]}')
    plotly_data.append(pid_traj)
    
    temp_df = traj_df[(traj_df.clip_name=='brokovich') & (traj_df.pid==pid) & (traj_df.time==clipdata_df[clipdata_df.clip_name=='brokovich']['clip_len'].iloc[0]-1)]
    pid_traj = go.Scatter3d(
        x=temp_df['x'],
        y=temp_df['y'],
        z=temp_df['z'],
        customdata=temp_df[['time','pid']],
        mode='markers',
        marker={'size':8, 'color':'red'},
        opacity=0.5,
        name='brokovich',
        legendgroup='brokovich',
        showlegend=False,
        hovertemplate='x: %{x:.3f}<br>y: %{y:.3f}<br>z: %{z:.3f}<br>t: %{customdata[0]}<br>pid: %{customdata[1]}')
    plotly_data.append(pid_traj)
    



# formatting
plotly_layout = go.Layout(showlegend=True, 
                          autosize=False,
                          width=800, 
                          height=600,
                          margin={'l':0, 'r':0, 't':35, 'b':0},
                          legend={'orientation':'h',
                                  'itemsizing':'constant',
                                  'xanchor':'center',
                                  'yanchor':'bottom',
                                  'y':-0.055,
                                  'x':0.5},
                          title={'text':'Mean-Unlike Trajectories for \"oceans\" and \"brokovich\"',
                                 'xanchor':'center',
                                 'yanchor':'top',
                                 'x':0.5,
                                 'y':0.98})

plotly_config = {'displaylogo':False,
                 'modeBarButtonsToRemove': ['resetCameraLastSave3d','orbitRotation']}

fig_traj = go.Figure(data=plotly_data, layout=plotly_layout)
fig_traj.show(config=plotly_config)

These trajectories are completely different from the mean trajectories. They begin at an entirely different direction and continue to travel in a fashion that is seemingly unrelated to their respective mean trajectories. However, it is important to remember that these mean-unlike trajectories are few in quantity in comparison to mean-unlike trajectories.

1.1. Divergence of a Single Trajectory

1.1.1. Total Divergence

# compares adjacent distances
def divergence_v1(traj):
    length = traj.shape[0]
    total_dist = 0
    for i in range(1,length):
        total_dist += sqrt((traj[i,0]-traj[i-1,0])**2 + (traj[i,1]-traj[i-1,1])**2 + (traj[i,2]-traj[i-1,2])**2)
    
    inc = 3 # increment = number of time steps to move forward
    dist = np.zeros(length-inc)
    for i in range(inc,length):
        dist[i-inc] = sqrt((traj[i,0]-traj[i-inc,0])**2 + (traj[i,1]-traj[i-inc,1])**2 + (traj[i,2]-traj[i-inc,2])**2)
    
    total_div = 0
    for start in range(inc,2*inc):
        div = 0
        for i in range(start, length-inc, inc):
            div += abs(dist[i] - dist[i-inc])
        total_div += div
    
    return(total_div / (inc*total_dist))



# compares all distances
def divergence_v2(traj):
    length = traj.shape[0]
    total_dist = 0
    for i in range(1,length):
        total_dist += sqrt((traj[i,0]-traj[i-1,0])**2 + (traj[i,1]-traj[i-1,1])**2 + (traj[i,2]-traj[i-1,2])**2)
    
    inc = 3 # increment = number of time steps to move forward
    dist = np.zeros(length-inc)
    for i in range(inc,length):
        dist[i-inc] = sqrt((traj[i,0]-traj[i-inc,0])**2 + (traj[i,1]-traj[i-inc,1])**2 + (traj[i,2]-traj[i-inc,2])**2)
    
    total_div = 0
    for i in range(len(dist)-1):
        for j in range(i+1,len(dist)):
            total_div += abs(dist[i]-dist[j])
    
    return(total_div / (inc*total_dist))

Now let’s make sure our definition of divergence makes sense for some defined example trajectories.

Work in progress (currently priority #2)

def test_divergence(name):
    if (name=='line_constant'): # expected divergence = 0
        length = random.randint(50,100)
        traj_arr = np.zeros((length,3))
        segment_len = random.randint(0,10)
        direction = np.random.rand(3)
        for i in range(1,length):
            for j in range(len(direction)):
                traj_arr[i,j] = traj_arr[i-1,j] + segment_len*direction[j]
                
    elif (name=='line_random'): # expected divergence = 0
        length = random.randint(50,100)
        traj_arr = np.zeros((length,3))
        direction = np.random.rand(3)
        for i in range(1,length):
            segment_len = random.randint(0,10)
            for j in range(len(direction)):
                traj_arr[i,j] = traj_arr[i-1,j] + segment_len*direction[j]
                
    elif (name=='line_increasing'): # expected divergence = 0
        length = random.randint(50,100)
        traj_arr = np.zeros((length,3))
        direction = np.random.rand(3)
        segment_len = random.randint(0,10)
        for i in range(1,length):
            for j in range(len(direction)):
                traj_arr[i,j] = traj_arr[i-1,j] + segment_len*direction[j]
            segment_len = segment_len * 1.5
        
    elif (name=='semicircle'): # expected divergence = 0
        length = random.randint(50,100)
        traj_arr = np.zeros((length,3))
        radius = random.random()*50
        for i in range(length):
            traj_arr[i,0] = radius * cos(pi*i/length)
            traj_arr[i,1] = radius * sin(pi*i/length)
        
#     elif (name=='curve'):

#     elif (name=='zigzag'):
        
    return divergence_v1(traj_arr)

print(f"Divergence of line with constant velocity: {test_divergence('line_constant'):.3f}")
print(f"Divergence of line with random velocity: {test_divergence('line_random'):.3f}")
print(f"Divergence of line with increasing velocity: {test_divergence('line_increasing'):.3f}")
print(f"Divergence of semicicle: {test_divergence('semicircle'):.3f}")

Divergence of line with constant velocity: 0.000
Divergence of line with random velocity: 0.388
Divergence of line with increasing velocity: 0.495
Divergence of semicicle: 0.000

temp_df = traj_df[(traj_df.clip_name=='overcome') & (traj_df.pid==1)]
single_traj = np.array(temp_df[['x','y','z']])
print(f"Divergence of single trajectory: {divergence_v1(single_traj):.3f}")

mean_traj = np.array(temp_df[['mean_x','mean_y','mean_z']])
print(f"Divergence of mean trajectory: {divergence_v1(mean_traj):.3f}")

Divergence of single trajectory: 0.365
Divergence of mean trajectory: 0.290

1.1.2. Divergence as a Function of Time

Work in progress (currently priority #1)

Need to define a function and create a figure to show how it works. Then I can start testing on defined examples.

1.2. Divergence Between Two Trajectories

Work in progress (currently priority #3)

Need to define a function and create a figure to show how it works.

2. IMDb with LSTMs